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(57) A method of embedding dgitat data in a source 
image includes the steps of: a) generating a mutti-level 
dalB image representing the digital data: b) convolving 
the multi-level data image with an encoding carrier 
image to produce a frequency dispersed data image; 
and c) adding the frequency dispersed data image to 
the source image to produce a source image containing 
embedded data. The data is recovered from the image 
by: a) cross correlating the source image containing 
embedded data with a decoding carrier image to 
recover the data image: and b) extracting the dgitat 
data from the recovered data image. 
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Description 

HELD OF THE INVENTION 

5 Tlie inventicxi relates generally to the field of image processing, and in particular to a method of enft>edding digital 
information in an image. 

BACKGROUND OF THE INVENTION 

10 tt is known that photographs or images can have information contained within the image area (for example: bar 
codes, date and time), tt is also known that digital information (computer readable) can be applied to the image area. 
There are have been several endeavors to embed digital data in images, for example for copyright protection. For van;, 
ous reasons as noted below, none of these methods are appropriate for embedding data in images containing general 
photographic content with minimal visibility while retaining the existing quality of the image. It is also a desirable feature 

15 to be able to encode the data in the image either optically or digitally. The techniques that do address embedding data 
in images are limited to special subsets of images including uniform fieUs of color, graphically generated images witti 
large areas of uniform color (such as the pie-charts of business graphics), or text. Known prior art methods of embedr 
ding digital information in images have several drawbacks for application to images having photographic pictorial con- 
tent. With these technologies, the inrtage may be partially or completely distorted, the embedded data can be digitally 

20 removed, or the method may require a significant nrxxtif ication to hardware in order to implement the technology. Some 
of these technologies use areas outside of the image area (glyphs or bar codes) because the techniques are largely 
destructive of the image itself and require a uniform background to read the infamation. Some methods aeate visible 
distortions within the image that can be unsightly and/or undesirable. Other techniques such as applying logos, labels, 
or stickers to an image can be easily hidden (covered) and can also be unsightly. Magnetic strips applied to the image 

25 require the scanning devices to be specially modified with a magnetic detector to allow the strip to be sensed. Most of 
these methods provide a binary solution (that is. they allow only a yes/ho or go/no-go solution). 

U.S. Patent 4.939.515 issued July 1990 to Adelson shows a technk^ue fa embedding digital information in televi- 
sion signals. The technk^ue first deconposes the image into a resolution pyramid, the layer of the pyramid having the 
highest spatial frequencies is omitted and a Hadamard-based bit stream is substituted. This approach relies on the tact 

30 that the optical spot of a CRT contributes tar more strongly to the high frequency attenuation of the CRT MTF than any 
of the transmission components. Therefore, the signal can be encoded, transmitted, and decoded by the electronics, 
while the optics of the CRT attenuates the high frequencies such that the embedded digital stream is not visibly objec- 
tionable. It also relies on the property of human visual psychophystcs where the high spatial frequency sensitivity of the 
visual system is limited to low temporal frequencies, and thus by having the high frequency digital stream change from 

35 TV f iekJ to TV field, the human eye is not very sensitive to it This method can achieve data embedding that is entirely 
invisible under some circumstances (i.e.. on TV monitors with poor MTF characteristics, far viewing distances, or cer- 
tain image sequences). Th's approach also requires the encoding step to be performed digitally. Another shortcoming > 
is that the technk^ue relies on systems with low physical MTF retative to the nyquist frequency, whk:h is generally not 
true aaoss the widely varying image qualities of current imaging systems. Yet another shortconrting is that it relies on 

40 the high frequency digital signal information having a high temporal frequency. This is impossUe to achieve in still 
images in hard-copy form. 

U.S. Patent 4.972.471 issued Novenrt)er 1990 to Gross uses notch filters and temporal modulation of their output 
in order to embed digital informatk)n inaudibly in recordings. One of the stated af^ications is toward the monitoring of 
broadca^ recordings for copyright& The recorded signal is embedded in a two pass algorithm. Rrst the recording is 

45 scanned and decomposed into s^eral notch frequencies (frequencies corresponding to the nnisical diatonic scale are 
suggested) to look for the best ptacement of a start code. This is deckled by use of the properties of audto masking 
known from the science of acoustic psychophysics, and the method looks for the presence of two neightXDring frequen- 
cies, one having a suffk:iently higher amplitude than the other. Audio masking dictates that the lower amplitude fre- 
quency can be renKved without the human observer being able to hear the difference. Once this temporal location is 

50 deckled in the first pass, the second pass of the algorithm determines the temporal starting point of the code and the 
notch frequencies used and calibrates this against the existing parallel SMPTE code. This data is stored t>y tho^e wish- 
ing to test kjentity of the signal (i.e.. the owners), and is not made available to those wishing to conxtpt the embedded 
data (i.e.. the copyright infringer). The second pass actually embeds the start code pattern and the Mentifying informa- 
tion in a specified temporal gap after the appearance of the start code. The bits of the klentifying information are 

55 encoded as temporal modulation of the notch filter content. Although the patent suggests that the method may be 
applied to images, the drawt>acks in trying to apply this approach to images include: 

1. there is no equivalent of the SMPTE time code in images and therefore finding an equivalent of the start code 
would be very difficult: 
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2. the placement of the embedded information depends on the content of the recorded signal in order to determine 
where (both in frequency and in time) to place the embedded information, which requires a digital or complex ana- 
log system: and 

3. the data is not spatially spread throughout the image. Consequently, if the start code or the identifying code is 
5 acddentalty blocked by unintentional or intentional corruptions of tiie signal, which in the optical imaging application 

would include such defects as dust and scratches, the data will be lost. 

US. Patents: 5.091,966 issued February 1992 to Bloomberg; 5,128.525 issued July 1992 to Stearns, et al.; 
5.168.147 issued December 1992 to Bloomberg; 5,221 ,833 issued June 1993 to Hecht; and 5.245.165 issued Septem- 

10 ber 1993 to Zhang, comprise a family of patents that address embedding digital information invisibly within in hard-copy 
images. However, they are limited to images containing signrficarrt uniform areas of color, such as found in computer- 
generated graphics for business and science data preserrtation. or images containing text. The approach they have 
taken is to use small localized and non-overlapping gray-scale shapes to carry the bit information. The shapes are 
refened to as glyphs, and are small enough and closely packed so that they visually merge into a uniform f ietol when 

15 printed on a sufficiently high resolution device (300dpt). This technique works in electrophotographic copying machines 
because such machines have the ability to capture very high resolution edges (the amplitude dimension is often cap- 
tured with low resolution, however) in order to keep the edges of the text sharp in appearance. TTie various patents 
address different shapes, different methods of dealing with scale changes, and different methods of dealing with image 
skew, such as rotation. The technique is basically a matched fitter method used to directly search the embedded image 

20 for the glyphs. The technique does not work with images consisting of photographic subject matter including such 
image features as texture and grain noise because the texture and grain noise woukJ mask.the detection of the glyphs. 

Several articles also desaibe various methods of embedding data for copyright protection. Schyndell, "A digital 
Watermark" IEEE ICIP-94 (1994) presents a method based on modulation of the least significant bit (LSB) of a digital 
image! This method requires digital encoding and the message would be lost if that bit is truncated, a if the noise levels 

25 in an image are inaeased. which is likely in the scanning of the digital image out to hard copy form, as well as scanning 
the image in from hard copy Sapwater et al.. "Eledronic Copyright Protection". Photo Electronics Imaging, Vol 7, No. 
6. pages 16-21 (1994) explores the issues in copyright protection of digital images, but does not present a satisfactory 
solution. The proposed solution is to digitally place a copyright symbol in tfie Yellow layer of a Cyan, Magenta, Yellow, 
and Black (CMYK) version of a color image. When the image is shown in color, the symbol is hard to see because of 

30 the weak Uue-yellow sensitivity of the human eye. However, when the color image is broken down into CM.Y. K layers, 
and the Y layer is displayed as a black and white image, it is possible for the human observer to see the copyright sym- 
bol. This technique has the disadvantages tfiat it requires the encoding to be performed digitally, and further requires 
human intervention to place the symbol in an area where it is not likely to degrade the image (such as avoiding the sub- 
ject s face). It also has the disadvantage that the detection of the copyright is not automated. Bender et al. Techniques 

35 for Data Hiding". SPIE Proceedings 2420 (1995) present two technques for embedding small amounts of digital data 
in images. One of these is termed Texture Block Coding and involves digitally copying a specifically shaped region of 
texture from one portion of the image to another. The bit pattern is encoded in the shape. Two disadvantages of this 
technique are tiiat the encoding must be done digitally, and further it requires a skilled image operator to select similarly 
textured regions to perform the swapi Further, the Bender paper does not explicitly state how to code digital information 

40 in the shapes, and has nc^ performed such studies. Another method proposed by Bender is called Patchwork and 
involves slight offsets off pixels (which may also bB low-pass filtered blob-like regions consisting of large numbers of 
actual pixels) in positive and negative directions so th^ as a specific path is taken through the image, the expected 
value of the differences deviates strongly from the mean of ttie image. This approach is intended to code only one bit 
of information, but it may be extended to a small number of bits (<8) by methods not explicitly defined in the paper. No 

45 way of implementing the technique optically or with hard copy was disclosed. A problem with applying the Patchwork 
technique to hardcopy is that of finding the proper pathway through an image after it has been converted to a digital 
image by scanning due to size and rotation issues. Walton. "Image Authentication for a Slippery New Age". Dr. Dobbs 
Journal. April, page 18-26 (1995) presents a technk^ue for data ent)edding tor image authentication using a checksum 
method. The technique is easily corrupted and would not likely survive a hardo^ form. The Walton technique has the 

50 further disadvantage that it requires the encoding to be performed digitally 

There is a need therefore for a technique for embedding digital data in images that can be implemented either dig- 
itally or optically, that will not visibly distort tiie image, is not easily corrupted by image content or defects, and is not lost 
when the image is cropped, rotated, or resized. 

55 SUMMARY OF T HE INVENTION 

The present inverrtion is directed to overcoming one or more of the problems set forth above. Briefly summarized, 
accading to one aspect of ttie present invention, a method of embedding digital data in a source image includes tiie 
steps of: a) generating a multi-level data image representing the digital data: b) convolving the multi-le/el data image 
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with an encoding carrier image to produce a frequency dispersed data iniage: and c) adding the frequency dispersed 
data image to the source image to produce a source image containing embedded data. The data is recovered from the 
image by: a) aoss correlating the source image containing embedded data with a decocfing carrier image to recover 
the data image: and b) extracting the digital data from the recovered data image. 
5 These and other aspects, objects, features and advantages of the present invention will be more clearly understood 
and appreciated from a review of the following detailed description of the preferred enrtxx£ments and appended claims, 
and by reference to the accompanying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

10 ^ . 

FIG. 1 is a flow chart illustrating generally the method of the present invention; 

FIG.. 2 is a flow chart showing the steps for converting digital data to a data image; ^ ... 

FIG. 3 is a flow chart showing the steps for generating an encoding carrier image; 
FIG. 4 is a graph showing a 1 -D linear chirp carrier signal: 
15 FIG. 5 is a graph showing a 1 -D log chirp carrier signal: 

FIG. 6 is a graph showing a 1 -D binary linear chirp carrier signal; 

FIG. 7 is a flow chart showing the steps for convolving the data image and the encodng canier image; 

FiG. 8 is a schematic diagram illustrating one method of adding the frequency dispersed data image to the source 

image: 

20 FIG. 9 is a schematic diagram illustrating an alternative method of adding the frequency dispersed data image to 
the source image; 

FIG. 10 is a schematic diagram illustrating a further alternative method of adding the frequency dispersed data 
image to the source image: 

FIG. 11 is a flow chart illustrating the steps of scanning an image having embedded data; 
25 FIG! 1 2 is a flow chart illustrating the steps of retovering the data image from a source image with embedded data: 
FIG. 13 is a flow chart illustrating one method for extracting digital data from a recovered data image; 
FIG. 14 is a flow chart illustrating an alternate method for extracting digital data from a recovered data image; 
FIG. 1 5 is a graph useful in desaibing the step for detecting the digital data in a data image; and 
FIGS. 16-21 are a schen^tic diagrams useful in desaibing the step for detecting the digital data in a data image. 

30 , 

To faciiitate understanding, identical reference numerals have been used, where possible, to designate identical 
elements that are contmon to the figures. 

DETAILED DESCRiPTIQN OF THE INVENTION 

35 

Referring to FIQ. 1 . the method of embedding digital data in an image according to the present invention will be 
described. The invention can be best considered in its two stages: an encoding process and a decoding process. Rrst. 
a multi-level data image is generated 10 from digital data. The digital data may represent an identifier for the image such 
as a catalogue nunnber. a copyright notice, information about the owner of the copyright such as her name and address. 

40 a pointer to additional inforn^on. a web site address on the Internet etc. The mutti-level data image as descnl>ed in 
detail below is an image having a constant background value and an array of spots on the background representing the 
digital data. The diata image may be produced using digital image processing techniques, or may be produced optically 
for example by exposing spots on a photographic film. 

Next the data image is convolved 12 with an encoding carrier image to form a frequency dispersed data image. 

45 The encoding canier image is preferably a rotationally symmetric, low amplitude, high frequency pattern enjoyed to 
spatially disperse the data image to mask its visibility when added to the source imaga Preferably the convolutton is 
performed on a digital computer using a well known Fourier Transform method and digital versk}ns of the data image 
and the encoding carrier image. The convolution may also be performed using a direct convolution algorithm on the cDg- 
ital computer. Alternatively, the convolution may be perfonned optically using well known optical convolution techniques 

50 and optical versions of the data image and the encoding carrier image. 

The frequency dispersed data image is then added 14 to the source image to form a source image with embedded 
data. As desaibed in more detail below, the addition may be performed either optically using tradttonal photographic 
processes, or digitally using a digital computer and digital versions of the source image and the frequency dispersed 
data image. If the addition is performed digitally, a hard copy version of the digital source image having embedded data 

55 may be produced using a digital printer such as a thermal, ink-jet. electrophotographic, or silver halide printer. 

The digital data is recovered from the source image having embedded data by first cross correlating 16. the source 
image having embedded data with a decoding can-ier image to produce a recovered data image. Preferably, the decod- 
ing carrier image is identical to the encoding earner image and the cross correlation is performed by a digital computer 
on digital versions of the source image having embedded data and the decocfing carrier image. If a hard copy of the 



EP0 777197 A2 



source image having embedded data is used, the hard copy is first scanned as deserved in detail below to produce a 
digital version. AlternatKrely. the cross con'elation may be performed using well known optical techniques and optical 
versions of the source image having embedded data and the decoding carrier image. 

Rnaliy. the digital data is extracted 18 from the recovered data image. Preferably, the digital data is extracted from 

5 a digital version of the recovered data image using a pattern recognition process in a digital computer. 

The details of step 10 for converting digital data to a Data image are shown in Rgure 2. The process begins with a 
message 20 in a 1 -D digital representation and first adds a unique identification code such as a start code 22. compris- 
ing a particular series of 1's and O's that will not occur in the actual digital message. The identification code need not 
be placed at the start of the data, it may be placed any where in the data, and may be dispersed within the data. An 

10 optional next step is to add any well known error con'ection coding 24 to the string. The particular error con-ection code 
chosen depends upon the application and factors such as bit error rate arxJ the anrx)unt of correlation in the errors. The 
new total number of bits are measured 26 and a geometric pattern sufficiently large to contain the total number of bits 
is chosen. The pattern can be any geometric shape, including a linear string or a rectangular array. In the following 
example, a square array will be desaibed. To determine the dimensions of the square anray. the lowest integer whose 

15 squared value is greater than or equal to the total number of bits is calculated in step 28. We will refer to this integer as 
n. and this value will be sent to the module for generating tfie Canier Image. The 1 -0 array is reordered 30 to an nxn 2- 
D array using an predetermined ordering method such as a raster pattern. 

The order of the following two steps is reversible. The data string is arranged into the nxn square array (if the data 
string does not fill the square an'ay. remaining positions in the array are padded, for example with zeros), with the pre- 

20 ferred method of filling the anay being lexicographical, although any other ordering method will work as long as the 
decoder knows the ordering scherrie as well. In step 32 the 0*s of the array are converted to values of -1 . This bipolar 
data representation scheme has advantages in the decoding step. After both of these steps 30 and 32. the next step 
34 is to resize the nxn array into an anray of size MxM. The value M of this new array is the size of the encoding carrier 
image as described below. This resizing 34 is accomplished by uniformly filling in the additional pixels with values of the 

25 backgrourxi level, so that an nxn array of pixels is embedded within an image of size MxM. The pixels of the embedded 
array contain multilevel data values such as 1 or -1 . These pixel locations will be refenred to as data centers. Optionally, 
this MxM image where the data centers are represented by single pixels may be convolved 36 with a data center shap- 
ing kernel such as a Gaussian or a pill box function to enlarge the data centers in the MxM array. This optional step may 
be used to facilitate the decoding procedure, which can then use a matched filter to aid in finding the data centers. The 

30 final result of this process is a data image 38 which has a digital size of MxM pixels in the preferred embodiment 

Referring to FIG. 3. the generation of the canier image will be described in detail. The size MxM of the Carrier 
Image is selected 40. where M is greater than n. The size M may be determined from other application design con- 
straints such as the size of the source image or the possibility that the source image will be cropped. Next a 1 -0 canrier 
signal of length M/2 and having a zero mean value is generated 42. One way of generating the canier signal is to create 

35 a sine wave whose frequency is varied from a specified low frequency to a specified high frequency. Such a sine wave 
of increasing frequency is referred to as a chirp signal. The frequency can vary linearly with position or logarithnrtically. 
and these signals will be refened to as linear chirps and log chirps, respectively. The choice of linear versus log and 
frequency ranges are determined for the particular application. The linear and log chirp signals are defined as follows: 

40 2 

Chirp, ^(r) n sin[2X • / min+^-1^^^^]]; and (1) 



45 



SO 



chirp]Qg[r)^s\Ti 




(2) 



55 



where roM/2 is ttie distance in units of pixels (everttually to become a radius, r). f^trn is the mininxjm frequency of the 
chirp, frnax is the maximum frequency of the chirp, and L is the total length of the chirp in the units of pixels. A graph of 
a linear chirp is shown in Rgure 4 where rs 256 pixels. Fmin^ 012S cycles^ixel. Fmax''0.333 cycles/jpixel. M a 512. A 
log chirp is shown in Rg. 5 where FminsO.01 25, and FmaxaO.400. It is also possible to quarrtize the chirps into a binary 
representation consisting of a square wave that changes in frequency as shown in Rgure 6. An entirely different type 
of carrier is a noise signal whose bandwidth is resti'icted to higher frequencies tor mininrtum human visibility. Optionally, 
a tapered weighting function may be used to modify tfiese 1 -0 carrier signals. 
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The next step 44 is to convert the 1 -D carrier signal into a 2-0 image. The preferred method is to rotationally sweep 
the 1 -D signal about the cenjer of the image (M/2. M/2) thus creating an efKXXjing canier image that is rotationally sym- 
metric akx>ut its center. This is done to enable the decoder to extract the data image regardless of the rotational position 
of the image when presented to the decoder. Other options for converting from a 1-0 signal to a 2-0 image include 

5 sweeping the 1-0 carrier signal through a limited range (e.g. 180*). An optional step 46 is to perform a spatial shift in a 
periodic fashion in order to renwve any positional bias in the decoder. For a signal swept about the image center, a spa- 
tial shift of xa-M/2 and y3-M/2 will reposition the 2-0 Carrier to the innage origin (x«0.y»0). This is useful if the decoder 
does not have the ability to find a start code in the 2-0 data array and one chooses to work in a very explidtiy defined 
closed system. The result of these steps is a MxM encoding canier image 48. 

10 Referring to FIG. 7. the process of convoMrig the encoding carrier image with the data image will be descrft}ed in 
more detail. The data image 38 and the encoding can-ier image 48 are convolved 50 to aeate an MxM frequency dis- 
persed data image 52. The convolution process is well known to those skilled in the art It is preferably acconplished 
by taking a 2-0 Oiscrete Fourier Transform (OFT) of both the encoding carrier image and the data image, multiplying 
the transforms together, and taking the inverse OFT of the product. The convolution may also be acconplished by direct 

75 spatial convolution. The resulting MxM frequency dispersed data image from the convolution is tiled 54 onto the source 
image. If the source image is not covered by an integer number of MxM tiles (generally the case), fractional portions of 
an MxM tile are used at the edges. In the case where the source image is smaller than the MxM frequency dispersed 
data image the reliability of en-or free recovery of the data is diminished. This may still be useful in applications where 
the mer9 presence of a message is important, such as in copyright notification. 

20 The final step 56 in this ^oc&ss is to scale the amplitude of the tiled frequency dispersed data image so that when 
added to the source image it is unnoticeable to the average human viewer. The scaling value depends on a nunrt>er of 
parameters of the image such as baseline noise level and displayed dynamk; range. We have determined the scaling 
values for 8-bit images on a series of film types, having quantified their inherent noise level, or grain (see Table 1). . 

25 . • 

Table 1 



Rim Type 


Estimated Scaling 
Value 


Kodachrome 25 


6 


Kodachrome 64 


8 


Ektachrome 100 


10 


Ektachrome 200 


16 



At this point the source image and the tiled frequency dispersed data image are added 14 (see FIG. 1) to produce 
the source image with embedded data. This addition can be done in a number of ways by either digital or analog proc- 
40 esses. In the digtal process, the digital versions of the images are the same size and are simply added together pixel 
by pixel. In areas where the sum is less than the mininujm or greater than the maxinum alk)wable code value, the sum 
is clipped. 

There are many applicatioris where it is convenient to add the images by analog methods such as in a photographic 
film or paper manufacturing process, in an optical photographic printer or in a camera. In this case, the resulting addition 

45 may not be a taie additton. but rather an addition subject to the noniinearity of the film or paper's tone scale curve. This 
addition essentially becomes an addition in the density domain (a log of the actual luminances) with some deviation at 
the toe and shouklers of the fOm or paper. Fortunately, these deviations only cause a slight reduction in the subsequent 
performance of the decoding step. In the optical addition case, a mask is made of the tiled frequency dispersed data 
image or a device is made to expose the tiled frequency dispersed data innage onto the film or photographic paper. In 

so either case. Table 2 lists the points in the optical pathway where the optteal addition may be performed. 



55 
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Table2 

1 . on the unexposed roil of film during or after the nianufacturing steps; 

5 

2. in the film plane of the camera; 

3. in the negative (or positive) gate of the eniarger or printer making prints from the film; 

4. as a contact print exposure either before or after the prints exposure to the image; 

10 5. as a simultaneous (a nearly simultaneous) exposure using a beam splitter in the machine printer; 

6. by pre-exposing the paper or film either during or after the manufacturing steps; or 

7. by including an additional layer containing the frequency dispersed data image that exposes the paper or film along 
with the source image exposure, the extra layer is washed away in the paper or f Dm's processing. 



FIG. 8 is a schematic diagram showing a photographic printer 58 wherein a photographic transparency 60 bearing 
the source image and a transparency 62 bearing a tiled frequency dispersed data image are mounted in the printing 

20 gate 64 of the photographic printer 58. The images are optically added by superimposing them in the printing gate and 
simultaneously exposing them onto photographic paper 66. FIG, 9 is a schenr^tic diagram illustrating a method of pre- 
exposing photographic paper with the tiled frequency dispersed data image. A transparency 68 bearing the til^ fre- 
quency dispersed data image is exposed by a light source 70 onto unexposed photographic paper 72 in the paper man- 
ufacturing process for producing photographic paper 74 with embedded data message according to the present 

25 invention. Referring to FIG. 10, a schematic diagram shows a reticle 76 bearing the tiled frequency dispersed data 
image in the film plane 78 of a camera 80. for exposing the tiled frequency dispersed data image onto the f Om simulta- 
neously with the photographic exposure. Alternatively, the adcfing step may be performed by affixing a transparency 
bearing the frequency dispersed data image onto the source image. 

FKa. 1 1 shows the steps involved in converting a hardcopy version of a source image having embedded data to a 

30 digital version of the image. Rrst the image is scanned 82 to produce a digital representation by standard means such 
as a flatbed scanner. 

A calibration 84 may be performed to calculate a pixel array RxR in the scanned image that is the same size in the 
scanned image that is occupied by the MxM frequency dispersed data image. R may be different from M due to differing 
pixel spadngs between the input scanner and the output scanner used to create the frequency dispersed data image. 
3$ due to optical magnification in the optical path, or due to digital size adjustment in the steps between the encoder and 
decoder. This step envoys the size M. the output pixel spacing (dpiout) of the device used to generate the frequency 
dispersed data image, the optical magnification of the printer employed to produce the hardcopy image (Mag), and the 
image scanner pixel spacing (dpijn) employed in step 82. R is calculated as follows: 

^ R»(M/clpi^)(Mag)(dpiJ (3) 

There are ateo several optional image processing steps following ttie scanning step 82. These include a conversion 
of a 3-color image to a luminance image 86. by using only one of the color records R. Q. or B, or any other technique 
that uses a weighted average of R. Q. and B. Another optional step is to enhance the high frequency component of the 

45 image 88 by either applying a high pass-filter or an unsharp mask to the image. 

A third optional step is to examine 90 the high frequency enhanced image to find the RxR pixel array with the lowest 
standard deviation within a specified mean gray-level range. The goal of these steps is to find an RxR pixel aray in ttie 
image where ttie performance of ttie decode will be optimized. An alternative to this third optional step is to sum the 
absolute values of the difference between the pixels and the local mean in a cancSdate RxR window of the high fre- 

50 quency enhanced image. This approach suffers lower performance but gains computational efficiency. These optional 
steps are intended to prevent a section of the image being seleded ttiat may be located in an area in the shadows of 
the image where tiie tone scale has reduced the contrast of the embedded frequency dispersed data image, or in tiie 
highlights where a similar contrast reduction occurs. Once this area is selected, the RxR pixel array is extracted 92 from 
the original unfiltered version of the scanned image. 

55 If the optional steps outiined above are omitted, any RxR pixel array may be selected from the image. The center 
of tiie image may be used as the default. Rnally. ttie RxR pixel array that has been extracted from the iniage is resized 
94 into an MxM pixel array 96 using conventional mettiods capable of non-integer resizing, such as bilinear interpolation 
or cubic spline interpolation. 

If ttie image having embedded data was an unmocfified digital image. R»M and ttierefore ttie steps 82. 84, and 94 
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would be omitted. The remaining steps would be performed with R=M . 

FIG. 12 shows the cros&K^orrelation step 16 (see FIG. 1>in further detail. TheMxM pixel array 96 and the MxM Car- 
rier Image 98 are aoss-correiated 100. The cross correlation is accomplished using well known digital image process- 
ing techniques. Analogous to the convolution step, the result of the cross<orrelation is an MxM pixel anray. The MxM 

5 pixel array resulting from the cross con-elation, referred to as the recovered data image, is then normalized 102 to a 
desired range, such as a 0-255 range for an 8-bit image. The recovered data image may be spatially shifted, rotated, or 
contain echoes of the original data image. 

Rgure 13 shows an **extreme value** method of extracting the 2-0 data array from the recovered data image 1 iO. 
For each row i of the MxM pixel array, the maximum and minimum pixel values are found, and their difference AH, is 

10 taken 1 12. refenred to as the horizontal range. The n highiest AH,- values are found and their i indices are stored 1 14. 
The integer n, as previously defined, is the number of data centers in a row of data centers. For each column j in the 
array, the maximum and mininrujm pixel values are found and their difference ^Vj is taken 1 1 8. referred to as the vertical 
range. The n highest values AVy are found and their indices j are stored 1 20. 

The next steps are optional and are intended to improve the performance when the recovered data image contairts 

75 strong echoes of the data image and/or the data centers are of low amplitude. The first of these optional steps 1 1 6 and 
122 are to determine the average spacing p\ and the standard deviation oi of the n indices i that were stored in the pre- 
vious step, where: 

Hi and (4) 



25 



n-1 



(5) 



The standard deviation at is compared with a first threshold value 124. If it is less than the threshold value, the 
selected values of i are uniformly spaced, and it is likely that they will represent the rows on which data centers are 
30 located. If however, the d value exceeds the threshold, this means that the lines are not evenly spaced and therefore 
at least one or more of the indices do not represent the k>cation of a row of data centers. In that case, a refining step 
134 is needed. The value \^ that contributes most to the high value of d is identified. The process returns to step 114 
and selects another n highest values of AH, . with 

excluded from selection. This process is repeated until d is lower than the threshold. 

The same procedure is applied to the columns of the image as shown in steps 122 and 126. When the process is 

40 complete for rows and columns, the average spacings are compared in step 1 28. If the absolute value of (|ii-Mj) is less 
than a second threshoM value, the selected values of i and j are uniformly spaced, and it is likely that they will represent 
the rows and colunms on which data centers are located, and the procedure moves to step 136. If not, the conclusion 
is reached 130 that there is no embedded message in the image. If these optional steps are omitted, then the procedure 
directly advances to step 136 from steps 114 and 120. 

45 At this point the data center locations are determined from the nxn intersections of the n selected values of both i 
and j. in ^ep 136. The next step is to determine the cross-correlation values located at or around these data centers. 
An optional step 138 is to filter or weight the local neighborhood of pixels at the data center by a sharpening kernel to 
enhance the cross-conetation value at the data center. The center of the range of all the cross-conetation values in the 
recovered data image is determined, the ac^-correlation values at the data centers are then compared 140 to the 

so center of the range (i.e.. 127.5 for a 0-255 range). If the aoss-con'etation value at the data center is greater than center 
of the range, the bit located at the data center is assigned to be 1 , and if less, the bit is assigned 0 in steps 1 42 and 1 44. 
respectively These bits are then loaded into an extracted nxn array 1 46 in direct conespondence to their location in the 
recovered data image. 

Figure 14 shows an alternate pattern matching method for extracting the nxn data array from the recovered data 
55 image 1 10 while accounting for possible rotation between the original and final images. Rrst an array spacing d is cal- 
culated as da M/n 1 50 . Each row of the recovered data image is processed as follows. The first d^l pixels are exam- 
ined 152, and if for any of these pixels, the absolute value of the pixel minus the center of the range cross-correlatjon 
values is greater than a predetermined threshold ta (see FIG. 1 5), the pixel is considered a possible first identified data 
center in an nxn array. In which case, the other (n-1) pixels in the line at a distance of integer multiples of d away from 
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the first identified pixel (the one identified in step 152) are examined. If at least n - 2 of these pixels meet the same ai- 
terion 154 for selecting the first pixel in the line, then declare this line a potential data line and save its y-position and 
the xiX)sition of the first pixel on the line 156. As shown in the example in FIG. 16, assuming n o 3. data center pixels 
are shown as f 3led circles and non-data center pixels are shown as open circles. On the first line all three potential data 

5 centers exceed the threshold, so the line is declared a potential data line. Only one point matches in ttie second line 
because the starting point chosen was not in fact a data point. The test is repeated using the next pixel that falls outside 
the threshold In the third line, two of the three pixels matched, so this line is also declared a potential data Hne. Allowing 
a match even when fewer than n pixels have been identified in a line improves performance In the presence of noise. 
The previous steps are repeated 158 until the entire recovered data image has been processed. 

10 Depending on the signal-to-noise ratio of the image, this procedure will yield a varying number of potential data 
lines. This information is employed to compute 160 the vertical spadngs between potential data lines Ai. the horizontal 
spadngs between potential data centers Aj. the mean of the vertical spadngs jxi, the mean of the horizontal spadngs 
^ij. the standard deviation for the vertical spadngs oi, and the standard deviation for the haizontal spadngs oj. If more 
than n potential data lines have been identified in step 162. some of them must have been classified incorrectly. In this 

75 case, each line is tested 166 to see if there is another potential data line some multiple of d lines away, either above or 
below in the recovered data image. For example, each of the lines in figure 1 7 is the same distance from each other tine 
except for the dashed line. Since the dashed line has no neighbors a multiple of d lines away, it Is deleted. If more than 
n data lines still remain step 166 is repeated. Next, the starting pixel positions on each line are compared. Any line with 
a starting pixel that does not match the starting pixel of at least one other line is discarded (this is similar to the process 

20 in step 1 66. but with a stronger constraint). 

If fewer than n data lines remain after step 1 60, one or more data lines were not found using the matching process 
of steps 1 52 and 1 54. When this condition occurs, additional data lines are assumed to exist at distance d from one of 
the identified data lines and are added to the set of data lines 164. For example, as shown in figure 18. assuming n 3 
4. only the first, second, and fourth data lines were located by horizontal matching. Since one line is known to be miss- 

25 ing. it is added In (dotted line) between lines 2 and 4. If exactly n data lines remain In step 162. then the standard devi- 
ations oi and oj are compared to a predetermined threshold t4 to ensure that the spadngs of the data centers are 
uniform. When the data centers have been thus identified, the data is extracted from the recovered data image as 
desaibed previously. That is. the known line positions, starting x-positions. and grU spadng are used to index into the 
original image and retrieve the code values of the data pixels at those locations 138-146. The pattern matching 

30 approach has the advantage over the extreme value method in that it Is usable in lower sigial-to-noise ratio images and 
is somewhat angle-Invariant, but has the disadvantage of being slower. 

If either of the standard deviations ci or oj are above the threshold t4. the recovered data image Is rotated 170 
through a small angle and the pattern matching process is repeated. If. after searching through a range of angles 1 72 
(e.g. 0-45 deg.), no standard deviations pass the threshold test of step 168. the process ends up at step 174, which is 

35 the condusion that no data message is present. In copyright applications, this may indicate that a notfce of copyright ^ 
has not been embedded in the image using the method of the present invention. 

The final step, as shown in FIGS. 19-21. is to convert the extracted nxn decoded 2-D data an-ay 1 84 of bits into the 
n^ data string. Although the method of the present invention requires that the recovered data image be of the same size 
as the tiles (I.e. MxM) of the embedded frequency dispersed data images, it does not require that their boundaries 

40 match. Therefore, the extracted nxn array of data will mo^ likely be spatially offset in a periodic fashion and possibly 
rotated through an integer multiple of 90 degrees. To recov^ the encoded 2-D data array 180, the decoded 2-D data 
array 184 is tiled as shown in FIG. 22 to produce a 2-D tiled data array 186. A 2-D tiled data array 186 is searched for 
the unique start code sequence 182 through integer nruiltiples of 90 degrees. Once the start code is found the rest of 
the 2-D nxn data anay 188 is extracted. whk;h contains the same bit pattern as the encoded data anay 180. The start 

45 code is used to correctly orient the data of the data array. 

If the hardcopy image was a transparency, it may be necessary to f Op the data array about one of Its axis and repeat 
the search with tour angles. If the particular application could residt in tone scale inversions, then It will be necessary to 
invert alt the bits and repeat the search tor the start code. 

The invention has been described with reference to a prefen^ed errtoocfiment However. It will be appredated that 

50 variations and modifications can be effected by a person of ordinary skill in the art without departing from the scope of 
the invention. 

55 10 generate multi-level data image step 

1 2 convolve data Image with encoding carrier image step 
1 4 add frequency dispersed data image to source image step 
16 cross conelate step 
18 extract data step 
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20 1-0 data message string 
22 add identification code step 
24 add error correction bits st^ 
26 measure total length of string step 
28 calculate integer n step 
30 reordering step 

32 convert bits to mufti-level form step 
34 resize step 

36 convolve with data center kernel step 
38 data image 

40 select carrier image size step 
42 generate 1 -D carrier signal step 

44 convert 1-0 canier signal to 2-0 encoding canier image step 
46 perform spatial shift step 
48 encoding canier image 
50 convolve step 

52 frequency dispersed data image 

54 tiling step 

56 scaling step 

58 photographic printer 

60 photogr4)hic transparency bearing source image 

62 photographic transparency bearing frequency cfispersed data image 

64 printing gate 

66 exposed photographic paper 

68 contact transparency with frequency dispersM data image 
70 light source 

72 unexposed photographic paper 

74 photographic paper with ent)edded data 

76 reticle 

78 film plane 

80 camera 

82 scanning step 

84 spatial calibration step 

86 RGB to luminance conversion step 

88 high pass fitter step 

90 find best RxR pixel area step 

92 extract RxR area step 

94 resize RxR image area step 

96 MxM subsection of image 

98 MxM earner intage 

100 cross correlation mep 

102 normalization step 

1 10 recova'ed data image 

112 store horizontal difference step 

114 store horizontal indices step 

1 1 6 find average horizontal spacing step 

1 1d store vertical (fifference ^ep 

120 store vertical indices step 

122 find average vertical spacing step 

1 24 horizontal conpare step 

1 26 vertical compare step 

1 28 compare average spacing step 

1 30 conclusion of no message step 

1 36 find data center step 

138 weigh local area step 

1 40 determine value step 

142 bital step 

144bit=i0 step 

146 create array step 
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150 calculate grid spacing step 
1 52 examine first d+1 pixel step 
1 54 check for grid step 
156 save grid locations step 
158 repeat step 

160 compute spacing statistics step 

1 62 compare number of grid points step 

1 64 find candidate line to add step 

1 66 find candidate line to omit step 

1 68 threshold test step 

170 rotate step 

172 last rotation step 

1 74 no message present step 

1 80 encoded 2-0 data array 

182 unique start code sequence 

1 84 decoded 2-D data array 

186 tiled 2-D data an-ay 

188 2-D message within tiled data array 



1 . A method of enft}edding digital data in a source image, comprising the steps of: 

a) g^erating a multi-level data image representing the digital data; 

b) convolving the multi-level data image with an encoding carrier image to produce a frequency dispersed data 
image; and 

c) adding the frequency dispersed data image to the source image to produce a source image containing 
embedded data. 

2. The method claimed in daim 1 . further comprising the steps of: 

a) aoss correlating the source image containing embedded data with a decoding carrier image to recover the 
data image; and 

b) extracting the digital data from the recovered data image. 

3. The method claimed in daim 1 . wherein the step of generating the multi-level data image, comprises the steps of: 

a) appending an identification code to the digital data: 

b) arranging the digital data in a 2-dimensionai array; 

c) generating a multi-level image having data centers representing the bits in the 2-dimensional data array; and 

d) e)q>andtng the multi-level image and filling in the spaces between the data centers with null level values in 
the nrtilti-levei image. 

4. The method datmed in daim 3. further comprising the step of adding ennor conrection bits to the digital data. 

5. The method claimed in daim 3. wherein the data centers are convolved with a shaping kernel to fadlitate sut>se- 
quent detection of the data centers. 

6. The method daimed in daim 1 . wherein the encoding carrier image is generated by the steps of: 

a) providing a 1 -dimensional carrier signal: and 

b) converting the 1 -dimensional carrier signal to a 2-dimensional carrier signal using a rotational sweep. 

7. The method daimed in daim 6. wherein the 1 -dimensional carrier signal is a linear chirp signal defined by: 
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where r is the distance in units of pixels. is the minimum frequency of the chirp, f^^x 'S the maximum frequency 
of the chirp, and L is the total length of the chirp in the units of pixels. 

8. The method daimed in claim 6. wherein the 1 -dimensional carrier signal is a logarithmic chirp signal defined by: 



10 



chirpiQg (r) = sin 2n 



In 



^min J J 



15 



9. 
10. 



20 



25 

11. 

12. 
30 13. 
14. 



where r is the distance in units of pixels, f^jn is the minimum frequency of the chirp, f^ax 'S the maximum frequency 
of the chirp, and L is the total length of the chirp in the units of pixels. 

The method daimed in daim 2. wherein the decoding carrier image is identical to the encoding earner image. 
The method daimed in claim 2. wherein: 

a) the decoding carrier image is generated by the steps of: 1) providing a 1 -dimensional carrier signal, and ii) 
converting the 1 -dimensional carrier signal to a 2-dimensional carrier signal using a rotational sweep: and 

b) the encoding carrier image is generated similarly by errptoying a subset of the rotational sweep used to gen- 
erate the decoder carrier image. 

The method daimed in claim 1 . wherein the step of convolving employs a periodic 2-D convolution. 

The method daimed in daim 1 . wherein the adding step is performed optically 

The method daimed in claim 2. wherein the steps are performed on a digital conputer. 

The method claimed in claim 1 . further comprising the step of repeating the frequency dispersed data image in a 
tiled fashion prior to the adding step. 



35 15. The method daimed in daim 2. further conprising the step of locating a subset of the source image containing 
embedded data having the lowest high-frequency compon^ variance and cross con-elating the subset with the 
decoding carrier image. 



40 



16. The method daimed in claim 2. wherein the aoss correlation step is periodic cross con-elation. 

17. An image having embedded digital data produced according to the method of daim 1 . 

18. The method daimed in daim 1 . wherein the step of adding is performed by optical techniques. 

45 19. The method daimed in daim 18. wherein the frequency dispersed data image is added to the source image in a 
photographic printer by pladng a transparency having the frequency dispersed image in the film gate of the printer. 

20. The metiiod daimed in daim 18. wherein the frequency dispersed data image is added to the source image in a 
photographic printer by pre-exposing photographic paper used in the printer with the frequency dispersed image. 



50 
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21. A photographic paper having exposed thereon a frequency dispersed data imaga 

22. The method daimed in daim 18. wherein the frequency dispersed data image is added to the source image in a 
photographic camera by pre-exposing photographic film used in the camera with the frequency dispersed image. 

23. A photographic film having exposed tiiereon a frequency dispersed data in^ge. 

24. The method daimed in claim 6. wherein the 1 dimensional carrier signal is a noise sequence. 
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25. The method claimed in claim 6» further comprising the step of applying a tapered weighting function to the carrier 
signal. 

26. The method daimed in claim 12, wherein the step of adding the frequency dispersed data image to the source 
5 image utilizes a scaling factor dependent on film characteristics. 

27. The method claimed in claim 12. wherein the adding step is performed in the film plane of a photographic camera. 

28. The method claimed in claim 1 2. wherein the adding step is performed by contact exposing a transparency bearing 
10 the frequency dispersed data image onto photographic paper. 

29. The method claimed in claim 12, wherein the addng step is perfbrmed by simultaneously exposing the source 
image and the frequency dispersed data image onto photographic paper in photographic printer. 

IS 30. The method claimed in claim 12, wherein the adding step is perfbrmed by affixing a transparency bearing the fre- 
quency dispersed data image onto the source image. 

31 . The method daimed in claim 3. further compristng the steps of: 

20 a) cross correlating ttie source image containing embedded data with a decoding carrier image to recover the 

data image; and 

b) extracting the digital data from the recovered data image by: 

i) converting the recovered data image into a two dimensional data array; 
25 ii) periodically tiling the two dimensional data aray to produce a tiled two dimensional data array; and 

iii) searching the tiled two dimensional data array fa the identification code to determine the starting loca- 
tion of the two dimensional data array. 

32. The method daimed in daim 2, wherein the recovered data image has rows and columns of values, further com- 
30 , prising the Steps Of : 

a) finding the difference between the minimum and maximum values for each row of the recovered data image 
to produce row difference values; 

b) finding the difference between the minimum and maximum values tor each column of the recovered data 
35 image to produce column difference values; 

c) finding the n highest difference values for the rows and saving the indices of these rows, where n is deter- 
mined from the size of the multi-level data message; 

d) finding the n highest difference values for the columns and saving the indices of these columns, where n is 
determined from the size of the mutti-level data message; and 

40 e) locating data centers in the recovered data image by intersecting the saved rows and columns. 
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